Vertex Classification for Segmented Images

Depth estimation is an essential component for scene understanding. With this in mind, we take a novel approach to 3D depth reconstruction which leverages off of the highly structured nature of our environment. We believe that high level structural cues, such as planes, edges, and vertices play an important role in resolving depth ambiguities and attempt to evaluate the significance of such factors. This approach is partly inspired by the constraint satisfaction of Waltz’s algorithm in ”Understanding of Line Drawings with Shadows” (1975). Waltz was concerned with finding the correct 3D interpretation of a line drawing by using the geometric constraints which connect edges place on each other. But in applying this method to real images, a number of issues arise. It is not clear where the 3D discontinuities lie in the image and as a result we cannot use hard constraint satisfaction to solve for the true depth. We attempt to distill the basic intuition and capture the geometric relationships between adjacent parts of an image probabilistically and generate a global depth reconstruction using a CRF. This project is largely a theoretical exploration of this approach and here we will specifically discuss the subtask of vertex classification.